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I.  INTRODUCTION 


Department  of  Defense  (DOD)  spending  is  the  largest  portion  of  the  federal 
government’s  discretionary  budget  and,  according  to  the  Brookings  Institution,  accounts 
for  nearly  3%  of  the  United  States’  gross  domestic  production.  DOD  requirements  extend 
deep  into  the  vast  and  diverse  American  economy,  and  its  procurement  for  goods 
constitutes  nearly  5%  of  the  country’s  total  manufacturing  output  and  up  to  20%  of  U.S. 
research  and  development  expenditures  (O’Hanlon,  2015).  In  2013,  the  DOD  procured 
more  than  $300  billion  in  goods  and  services,  a  sum  larger  than  the  gross  domestic 
product  of  many  other  countries  (Government  Accountability  Office  [GAO],  2014).  The 
individuals  responsible  for  procuring  these  items  are  DOD  contracting  professionals. 
Contracting  professionals  are  charged  with  procuring  the  right  services  or  items  for  the 
right  price,  at  the  right  time,  in  keeping  with  the  public’s  confidence  and  in  accordance 
with  public  policy  goals  (Federal  Acquisition  Regulation  [FAR],  2016).  In  recent  years, 
with  a  tightening  DOD  budget,  DOD  stakeholders  are  scrutinizing  the  strategies  and 
methods  used  to  procure  what  the  DOD  needs.  Anticipating  an  era  of  lower  budgets,  the 
DOD  released  its  Better  Buying  Power  (BBP)  initiative — the  goal  of  which  was  to 
encourage  defense  professionals  to  do  more  with  less  and  to  focus  on  ways  to  become 
more  efficient  in  their  procurement  efforts  (Kendall,  2012).  Contracting  professionals  are 
expected  to  employ  procurement  strategies  to  purchase  requirements  at  best  value  to 
maximize  the  overall  benefit  to  the  DOD.  The  FAR  provides  guidelines  to  aid  contracting 
professionals  to  attain  the  best  value:  lowest  price  technically  acceptable  (LPTA)  and 
tradeoff  source  selection  strategies.  Contracting  professionals  may  use  LPTA  if  the 
requirements  are  well-defined  and  if  contract  outcomes  are  certain.  On  the  other  hand, 
contracting  professionals  may  use  a  tradeoff  strategy  to  leverage  the  trade  space  among 
schedule,  cost,  and  performance.  Ultimately,  the  employment  of  the  correct  best  value 
approach  will  support  the  BBP  initiative’s  goal  of  maximizing  efficiency  and  reducing 
costs  through  industry  competition  and  innovation  (FAR,  2016). 
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A.  PURPOSE 

The  purpose  of  this  research  is  to  determine  whether  a  relationship  exists  between 
the  contract  source  selection  strategy  used,  LPTA  or  tradeoff,  and  the  contract  pre-award 
and  post-award  performance  metrics.  We  address  the  following  metrics:  procurement 
administrative  lead-time  (PALT),  contractor  performance  ratings,  and  earned  value 
management  (EVM)  data  to  show  how  efficiently  contractors  are  performing  work  within 
budgeted  and  scheduled  constraints.  Our  intent  is  to  provide  contracting  professionals 
with  relevant  and  reliable  information  to  assist  in  selecting  source  selection  strategies. 

B.  RESEARCH  QUESTIONS 

This  research  intends  to  answer  the  following  questions: 

•  How  does  the  source  selection  strategy  affect  pre-award  metrics  (e.g., 
PALT,  number  of  solicitation  amendments  or  protests)? 

•  How  does  the  source  selection  strategy  affect  post-award  ratings 
(Contractor  Performance  Assessment  Reporting  System  [CPARS]  and 
EVM  performance  metrics)? 

C.  METHODOLOGY 

The  research  methodology  incorporates  a  literature  review,  data  collection,  data 
analysis,  and  discussion  of  our  findings.  The  literature  review  discusses  procurement 
statutes  and  regulations,  the  contract  management  process,  source  selection  strategies, 
and  a  review  of  investigative  reports,  defense  industry  perspectives,  and  ongoing  debates 
regarding  source  selection  strategies.  We  will  review  as  many  contract  files  as  possible  in 
a  four-day  period.  The  scope  of  review  will  be  limited  to  Navy  contracts  awarded  from 
2009-2015  and  managed  by  Naval  Sea  Systems  Command  (NAVSEA),  located  at  the 
Navy  Yard  in  Washington,  DC.  We  will  review  only  contracts  that  employ  LPTA  or 
tradeoff  strategies  using  FAR  Part  15.  We  intend  to  collect  data  from  contracts  exceeding 
$1  million  to  trigger  reporting  in  the  CPARS.  For  contracts  that  employ  a  tradeoff 
strategy,  we  investigate  award  criteria  to  determine  the  most  important  award  criteria 
selected. 
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D.  BENEFITS 


The  findings  of  this  research  can  inform  contracting  professionals  of  the  potential 
economic,  performance,  and  schedule  consequences  of  employing  a  particular  source 
selection  strategy.  The  benefit  of  conducting  this  research  is  to  encourage  contracting 
professionals  to  pursue  a  strategy  that  favors  and  maximizes  efficiency  and  productivity 
in  military  procurement  (Kendall,  2012).  This  research,  in  conjunction  with  prior  and 
future  projects  of  similar  scope  and  purpose,  could  serve  as  a  complement  to  and 
expansion  of  the  DOD’s  BBP  efforts  to  maximize  best  value  when  procuring  goods  and 
services. 

E.  LIMITATIONS 

The  time  allotted  to  review  contract  files,  insufficient  sample  size,  complexity  and 
organization  of  contract  files,  and  access  to  pertinent  data  are  all  key  limitations  to  the 
project’s  research.  We  were  allocated  four  days  to  review  as  many  contract  files  as 
possible.  Due  to  time  constraints,  we  collected  data  from  a  relatively  small  sample  size  of 
25  contracts.  We  were  not  afforded  additional  time,  which  would  have  allowed  us  to 
collect  data  from  a  larger  sample  size.  Due  to  the  complexity  of  some  contract  files,  we 
spent  additional  time  examining  several  volumes  of  documents  associated  with  one 
contract  to  locate  essential  data.  Furthermore,  not  all  contract  files  contained  a  contract 
file  checklist,  which  impeded  our  efforts  to  review  files  in  a  timely  manner.  Contract  files 
that  incorporated  a  checklist  helped  us  to  streamline  our  review  and  to  collect  data 
efficiently.  Access  to  data,  specifically  CPARS  and  EYM,  posed  a  significant  limitation 
to  the  project.  In  order  to  access  CPARS  data,  we  were  required  to  sign  non-disclosure 
agreements  with  program  offices  managing  specific  contracts.  Additionally,  EVM  data  is 
located  in  a  central  repository  managed  by  the  DOD’s  Performance  Assessment  and  Root 
Cause  Analysis  (PARC  A)  office.  In  order  to  access  EYM  data  through  PARC  A’ s 
database,  we  were  required  to  submit  an  application  and  request  permission.  The 
significant  amount  of  time  between  submission  of  the  application  and  access  to  the 
database  limited  our  ability  to  collect  EVM  data  in  a  timely  manner. 
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F.  ORGANIZATION 

This  report  is  organized  into  five  chapters.  Chapter  II  is  a  literature  review  that 
addresses  the  contract  management  process,  source  selection  strategies,  investigative 
reports,  industry  perspectives,  and  ongoing  debates  regarding  source  selection  strategies. 
Chapter  III  explores  the  organization  and  responsibilities  of  DOD  acquisition 
organizations,  Navy  acquisition  organizations,  and  NAVSEA.  Chapter  IV  provides  a 
description  of  the  data  collected,  data  analysis,  and  findings.  Chapter  V  presents 
conclusions  and  recommendations  for  further  research. 

G.  SUMMARY 

Chapter  I  introduced  the  research  topic  and  highlighted  the  purpose  of  the 
research,  research  questions,  methodology,  benefits  and  limitations  of  the  research,  and 
the  organization  of  the  report.  The  next  chapter  provides  a  literature  review  of  the 
procurement  statutes  and  regulations  relevant  to  the  contracting  process,  the  contract 
management  process,  source  selection  strategies,  investigative  reports,  industry 
perspectives,  and  the  ongoing  debate  regarding  source  selection  strategies. 
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II.  LITERATURE  REVIEW 


This  chapter  discusses  the  elements  of  the  contract  management  process  and 
examines  relevant  procurement  statutes  and  regulations,  and  the  ongoing  debate 
regarding  source  selection  strategies. 

A.  PROCUREMENT  STATUTES  AND  REGULATIONS 

The  vision  of  the  Federal  Acquisition  System  is  to  “deliver  on  a  timely  basis  the 
best  value  product  or  service  to  the  customer,  while  maintaining  the  public’s  trust  and 
fulfilling  public  policy  objective”  (FAR  1.102).  To  support  the  federal  acquisition 
system’s  policy  objectives,  the  FAR  addresses  or  implements  nearly  every  acquisition 
policy  or  statute  that  governs  how  the  federal  government  conducts  procurement 
activities. 

For  example,  the  Buy  American  Act  of  1933  requires  the  federal  government  to 
procure  supplies,  construction  material,  and  manufactured  goods  from  American 
suppliers.  The  intent  of  the  statute  is  to  give  preferential  treatment  in  the  distribution  of 
funds  to  domestic  suppliers.  However,  a  procurement  officer  may  acquire  foreign 
products  if  it  is  in  the  public’s  interest  or  if  the  item  is  unavailable  in  sufficient  quantity 
and  quality  in  the  United  States  (FAR,  2016). 

The  Small  Business  Act  of  1953  created  the  Small  Business  Administration 
(SBA),  an  independent  agency  designed  to  assist  and  advise  small  business  owners.  The 
SBA  provides  an  array  of  services  and  is  the  lead  advocate  for  small  business 
participation  in  federal  contracting.  The  Small  Business  Act  requires  “contracting  officers 
to  ensure  that  a  fair  portion  of  government  contracts  in  each  industry  category  are  placed 
with  small  businesses”  (FAR  19.502). 

The  Competition  in  Contracting  Act  (CICA)  of  1984  mandates  that  the  federal 

government  promote  and  provide  for  full  and  open  competition  prior  to  entering  into  a 

contractual  agreement.  The  FAR  requires  contracting  officers  to  “provide  for  full  and 

open  competition  through  use  of  the  competitive  procedure  such  as  sealed  bids, 

competitive  proposals,  combination  of  competitive  procedures,  or  other  competitive 
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procedures  ”  (FAR  6.102).  If  contracts  are  not  entered  into  competitively,  they  must  meet 
one  of  the  seven  exemptions  outlined  in  the  FAR: 

•  Only  one  adequate  source 

•  Unusual  or  compelling  urgency 

•  Mobilization  of  the  industrial  base 

•  International  agreements  or  treaties 

•  Authorized  or  required  by  statute 

•  National  security 

•  Public  interest 

The  Truthful  Cost  and  Pricing  Data  Act,  formerly  the  Truth  in  Negotiations  Act 
(TINA),  was  passed  in  1962  and  has  been  most  recently  amended  by  the  1987  DOD 
Authorization  and  Appropriations  Act  (Calhoon  &  Sybert,  2012).  In  the  absence  of 
competition,  the  Truthful  Cost  and  Pricing  Data  Act  allows  the  government  to  obtain 
certified  cost  or  pricing  data  from  a  contractor.  Certified  cost  or  pricing  data  allows  the 
government  to  conduct  an  independent  cost  analysis  to  determine  fair  and  reasonable 
pricing.  However,  the  Truthful  Cost  and  Pricing  Data  Act  does  not  apply  to  commercial 
items,  prices  set  by  law  or  regulations,  contracts  sought  when  adequate  price  competition 
exists,  or  contracts  at  or  below  the  simplified  acquisition  threshold  of  $750,000  (FAR, 
2016). 

The  Federal  Acquisition  Streamlining  Act  (FAS A)  of  1994  and  Federal 
Acquisition  Reform  Act  (FAR A)  of  1996  sought  to  reform  the  federal  government’s 
acquisition  system  and  streamline  procurement  functions.  The  Clinton  administration 
wanted  to  downsize  the  federal  acquisition  workforce  and  lower  procurement  obstacles 
by  expanding  the  definition  and  use  of  commercial  items.  Items  could  be  categorized  as 
commercial  if  technology  was  still  evolving  or  if  the  item  could  be  sold  to  the  general 
public  in  time  to  satisfy  the  government’s  requirement.  The  FAS  A  and  FARA  also  placed 
considerable  emphasis  on  conducting  market  research  and  commercial  item  procurements 
and  looked  to  gain  efficiencies  through  low-value,  high-volume  procurement 
transactions.  The  FASA  “signaled  a  dramatic  change  in  acquisition  policy  and  shifted 
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source  selection  strategies  to  be  made  on  a  best  value  basis  and  not  the  lowest  price” 
(Office  of  the  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
[OSD(AT&L)],  2016,  p.  iv).  The  best  value  continuum  allows  contracting  professionals 
to  consider  non-price  factors,  such  as  past  performance,  during  source  selections.  FASA 
also  expanded  opportunities  for  small  disadvantaged  businesses  and  woman-owned 
businesses  by  mandating  that  5%  of  contracts  be  awarded  to  small,  disadvantaged 
businesses  and  5%  to  small,  woman-owned  businesses  (Cohen  Seglias.  n.d.). 

The  origins  of  the  FAR  can  be  traced  back  to  the  Armed  Services  Procurement 
Regulation  of  1947;  however,  it  was  not  until  1984  that  the  FAR  was  codified  in  Title  48 
of  the  Code  of  Federal  Regulations.  The  FAR  was  implemented  by  Congress  to 
standardize  acquisition  policy  across  the  federal  government  and  to  eliminate  agency 
specific  procurement  procedures  (FindLaw,2016).  The  FAR  directs  the  acquisition 
process  by  which  the  federal  government  purchases  products  or  services  through 
contracting.  Contracting  is  defined  by  the  FAR  as  a  means  of  “purchasing,  renting, 
leasing,  or  otherwise  obtaining  supplies  or  services  from  nonfederal  sources.  Contracting 
includes  description  (but  not  determination)  of  supplies  and  services  required,  selection 
and  solicitation  of  sources,  preparation  and  award  of  contracts,  and  all  phases  of  contract 
administration”  (FAR  2.101).  The  contract  management  process  is  governed  by  the  FAR 
and  is  discussed  in  greater  detail  in  the  following  section. 

B.  CONTRACT  MANAGEMENT  PROCESS 

The  FAR  defines  a  contract  as  “a  mutually  binding  legal  relationship  obligating 
the  seller  to  furnish  the  supplies  or  services  and  the  buyer  to  pay  for  them”  (FAR  2.101). 
The  key  term  in  this  definition  when  trying  to  understand  the  significance  of  the  contract 
management  process  is  relationship.  This  relationship  begins  before  the  contract  is  even 
awarded.  When  developing  and  executing  a  contract,  the  government  and  its  prospective 
business  partners  encounter  risk  and  uncertainty.  The  management  and  avoidance  of  that 
associated  risk  and  uncertainty  is  one  of  the  primary  purposes  for  contract  management. 
In  most  cases,  risk  cannot  be  completely  removed  from  the  process,  but  with  a  sound  risk 
management  plan  and  communication  between  government  and  offeror,  the  risk  can  be 
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managed.  It  is  mutually  beneficial  to  both  government  and  offeror  to  act  together  in  this 
effort.  It  is  also  challenging  though,  as  the  ultimate  objective  of  the  government  is  the 
responsible  use  of  public  funds,  while  the  ultimate  objective  of  the  offeror  is  to  maximize 
profits  (Wright,  2007).  The  contract  management  process  serves  to  balance  those 
conflicting  objectives  while  managing  the  risk  associated  for  both  parties.  It  is  a  multi¬ 
faceted  process  demanding  technical,  risk  management,  financial,  and  communication 
skills  that  Garrett  (2007)  appropriately  describes  as  “the  art  and  science  of  managing  a 
contractual  agreement”  (p.  18).  Garrett  breaks  this  process  up  into  three  phases,  which  he 
calls  pre-award,  award,  and  post-award.  The  pre-award  phase  for  the  government 
includes  procurement  planning,  solicitation  planning,  and  solicitation.  In  addition,  for  the 
prospective  contractors,  it  includes  pre-sales  activity,  the  decision  to  bid  or  not,  and  bid 
or  proposal  preparation.  The  award  phase  includes  source  selection  for  the  government 
and  contract  formation  for  the  contractor.  Finally,  the  post-award  phase  includes  contract 
administration  and  contract  closeout  or  termination  for  both  the  government  and  the 
contractor.  Figure  1  displays  this  breakdown.  Each  of  the  three  phases  and  their 
associated  steps  are  described  in  detail  in  the  following  sections. 


Figure  1.  Contract  Management  Process.  Source:  Garrett  (2007). 
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1. 


Pre-Award  Phase 


The  pre-award  phase  is  critical  to  the  success  of  a  contract.  During  this  phase,  the 
government  identifies  its  requirement  and  then  determines  how  the  acquisition  team  can 
satisfy  that  requirement.  Critical  issues  such  as  cost,  schedule,  and  technical  requirements 
are  developed  by  the  government  and  then  communicated  to  industry  during  this  phase. 
The  success  and  precision  of  that  communication  is  essential  to  not  only  the  prospective 
contractors’  understanding  of  the  requirement,  but  also  in  their  decision  to  develop  a 
proposal  and  in  the  accuracy  and  completeness  of  that  proposal  (Garrett  &  Parrott,  2007). 

a.  Procurement  Planning 

Procurement  planning  begins  the  contract  management  process.  Before  a  contract 
can  be  awarded,  the  government  must  identify  a  requirement.  This  requirement  finds  its 
roots  in  what  Garrett  (2007)  calls  the  “make-or-buy”  decision.  Simply  stated,  this  is  the 
decision  for  the  government  to  fulfill  all  or  part  of  that  requirement  utilizing  government 
resources,  capabilities,  and  manpower,  or  to  outsource  all  or  part  of  that  requirement  to  a 
commercial  organization.  The  government  must  then  further  develop  and  refine  that 
requirement  into  a  statement  of  need,  technical  and  schedule  requirements,  cost  goals, 
and  capability  and  performance  thresholds  (Garrett  &  Parrott,  2007).  Once  the 
requirement  is  fully  defined,  market  research  must  be  conducted  to  determine  industry 
capabilities.  Market  research  is  a  critical  step  in  managing  the  risk  of  the  entire  contract 
management  process.  It  provides  a  snapshot  of  who  in  the  industry  may  have  the 
capability  to  meet  the  requirement,  if  there  is  already  a  commercial  item  available  to  meet 
the  requirement,  and  information  useful  for  cost  estimation  (Wright,  2007).  FAR  Part  10 
directs  government  contracting  professionals  to  conduct  market  research  and  identify 
potential  sources  in  an  effort  to  promote  full  and  open  competition. 

In  an  effort  to  “identify  and  resolve  concerns  regarding  the  acquisition  strategy, 
including  proposed  contract  type,  terms  and  conditions,  and  acquisition  planning 
schedules  as  well  as  the  feasibility  of  the  requirement,  and  any  other  industry  concerns  or 
questions,”  FAR  Part  15  encourages  government  agencies  “to  promote  early  exchanges 
of  information  about  future  acquisitions”  (FAR  15.201).  These  early  exchanges  can  be 
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accomplished  through  industry  “days’Vconferences,  pre-solicitation  notices,  requests  for 
information  (RFIs),  and  other  means.  In  determining  the  type  of  contract  to  use,  the 
associated  costs,  schedule,  and  performance  risks  must  be  considered.  Figure  2  shows  the 
relationship  between  risk  and  contract  type  (Garrett,  2007).  The  complexity  of  the 
requirement  is  often  a  large  factor  in  determining  the  appropriate  type  of  contract  and 
associated  share  of  risk  the  government  should  take  on.  For  example,  if  the  government 
requirement  is  a  non-developmental  commercial  item,  the  government  should  award  a 
firm  fixed  price  contract  and  place  the  majority  of  the  risk  on  the  contractor.  On  the  other 
hand,  if  the  government  requirement  is  to  develop  a  new  weapon  system  with  no 
technological  precedent,  it  may  be  necessary  for  the  government  to  share  or  even  take  on 
the  majority  of  the  associated  risk  through  the  use  of  a  cost  type  contract. 


Government/Buyer  Risk 


<- 

-> 

Fixed-price 

T&M  or  L/H 

Cost-reimbursement 

^ _ 

Types  of  Contracts 

1 

II  III 

\ 

1 

II  III 

s 

FFP  FP/EPA  FPI 

CS  CR  CPIF  CPAF  CPFF 

<- 

Contractor/Seller  Risk 

-> 

Figure  2.  Contract  Types  and  Associated  Risk.  Adapted  from  Garrett  (2007). 

Development  of  the  requirement  and  analysis  of  market  research  ultimately 
determines  the  direction  for  the  rest  of  the  contract  management  process.  The  choice  to 
use  full  and  open  competition  or  sole  source,  to  procure  a  commercial  item  or  require  the 
use  of  military  specifications,  and  to  make  the  contract  fixed  price  or  cost  type,  as  well  as 
the  choice  of  source  selection  strategy,  all  have  their  origins  in  the  analysis  of  the 
requirement  and  market  research.  Procurement  planning  lays  the  foundation  for  the  rest 
of  the  contract  management  process. 
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b.  Solicitation  Planning 

Once  the  government  fully  understands  its  requirements  and  the  industry 
conditions  and  capabilities,  it  must  prepare  the  document  that  communicates  those 
requirements  to  industry.  This  is  done  through  the  use  of  a  procurement  document,  or 
more  specifically  a  request  for  proposal  (RFP)  under  FAR  Part  15.  The  RFP  includes  the 
statement  of  work  (SOW),  and  the  terms  and  conditions,  and  directs  the  offeror  on  how  to 
respond  to  the  RFP.  Garrett  (2007)  highlights, 

Procurement  documents  should  be  structured  to  facilitate  accurate  and  complete 
responses  from  prospective  sellers  ...  and  they  should  be  rigorous  enough  to 
ensure  consistent,  comparable  responses  but  flexible  enough  to  allow 
consideration  of  seller  suggestions  for  better  ways  to  satisfy  the  requirements,  (pp. 
89-90) 

The  government  generally  establishes  the  evaluation  criteria  on  which  all  offerors’ 
responses  will  be  rated  during  this  step  as  well  (Garrett,  2007). 

c.  Solicitation 

FAR  Part  2.101  defines  a  solicitation  as  “any  request  to  submit  offers  or 
quotations  to  the  Government.”  During  this  step,  the  government  releases  its  requirement 
to  industry  in  an  effort  to  receive  multiple  competitive  proposals.  It  does  so  through  the 
use  of  an  electronic  portal  called  Federal  Business  Opportunities  (FedBizOpps). 
FedBizOpps  is  a  single  government-wide  point  of  entry  for  all  federal  government 
opportunities  over  $25,000  (Nash,  Schooner,  O’Brien-DeBakey,  &  Edwards,  2007).  The 
portal  allows  all  potential  offerors  from  industry  to  view  federal  requirements  and 
determine  if  they  have  the  capability  and  desire  to  meet  them.  Garrett  (2007)  emphasizes 
the  importance  of  the  solicitation  communicating  the  requirements  of  the  government 
clearly.  A  failure  to  do  so  often  results  in  “delays,  confusion,  fewer  bids  or  proposals,  and 
lower  quality  responses”  (Garrett,  2007,  pp.  24-25).  The  use  of  integrated  project  teams, 
comprised  of  multiple  senior  representatives  from  each  functional  department,  is  a 
technique  Garrett  (2007)  recommends  in  ensuring  the  government’s  requirements  are 
clearly  and  completely  articulated.  In  some  cases,  the  government  may  simply  choose  to 
provide  a  statement  of  objectives  (SOO),  rather  than  define  how  to  meet  requirements 
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using  an  SOW,  and  request  the  prospective  contractors  to  propose  their  recommendation 
to  meet  those  objectives  within  the  constraints  provided  (Garrett  &  Parrott,  2007).  In  this 
step,  it  is  also  important  for  the  government  to  ensure  it  has  not  included  terms  and 
conditions,  unrealistic  delivery  dates,  or  other  constraints  that  make  the  solicitation 
undesirable  to  potential  offerors.  Minimizing  constraints  encourages  maximum 
competition  and  reduces  the  risk  of  not  receiving  a  proposal  or  bid  and  delaying  the 
acquisition  process  (Wright,  2007).  Ultimately,  the  desired  outcome  of  the  solicitation  is 
to  receive  several  competitive  proposals  that  can  then  be  evaluated  based  upon  the  source 
selection  strategy  developed  during  procurement  planning.  The  award  phase  and  source 
selection  are  discussed  in  detail  in  the  following  section. 

2.  Award  Phase 

During  the  award  phase,  acquisition  officials  must  select  the  offer  that  presents 
the  best  value  to  the  government.  The  source  selection  process  can  be  as  simple  as 
choosing  the  lowest  priced  offer  that  meets  all  technical  specifications  or  as  in  depth  as 
evaluating  each  proposal  based  upon  multiple  evaluation  criteria.  This  process  includes 
the  negotiation  of  contract  terms  and  conditions  to  be  included  into  the  awarded  contract. 
The  award  phase  is  of  critical  importance  to  the  contract  management  process  as  it 
determines  the  government’s  business  partner  for  the  remaining  duration  of  the  process 
and  establishes  the  ultimate  source  of  the  government’s  requirement  (Garrett,  2007). 

a.  Source  Selection 

The  Source  Selection  Authority  (SSA),  with  assistance  from  the  Source  Selection 
Advisory  Council  (SSAC)  and  the  Source  Selection  Board  (SSB),  is  responsible  for 
selecting  the  source  that  offers  the  best  value  to  the  government.  FAR  15.303  states  that 
“the  contracting  officer  is  designated  as  the  source  selection  authority,  unless  the  agency 
head  appoints  another  individual  for  a  particular  acquisition  or  group  of  acquisitions.” 
The  SSA  appoints  senior  government  personnel  to  the  SSAC  and  SSB  to  advise  on  the 
conduct  of  the  source  selection  process  and  oversee  the  evaluation  of  proposals  submitted 
(Nash  et  ah,  2007).  This  collective  group  is  often  referred  to  as  the  source  selection  team. 
FAR  15.304  states  that 
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the  award  decision  is  based  on  evaluation  factors  and  significant  sub¬ 
factors  that  are  tailored  to  the  acquisition.  Those  evaluation  factors  and 
significant  sub-factors  must  represent  the  key  areas  of  importance  and 
emphasis  to  be  considered  in  the  source  selection  decision  and  support 
meaningful  comparison  and  discrimination  between  and  among  competing 
proposals. 

It  goes  on  to  state  that  agency  acquisition  officials  review  evaluation  factors  and 
determine  their  relative  significance,  but  that  “price  or  cost  to  the  Government  shall  be 
evaluated  in  every  source  selection  . . .  and  the  quality  of  the  product  or  service  shall  be 
addressed  in  every  source  selection  through  consideration  of  one  or  more  non-cost 
evaluation  factors”  (FAR  15.304).  Wright  (2007)  says  there  are  two  associated  risks 
during  the  source  selection  process:  proposal  risk  and  performance  risk.  Proposal  risk  is 
the  risk  associated  with  how  the  contractor  is  proposing  to  meet  the  requirement.  The 
government  source  selection  team  must  evaluate  each  contractor’s  technical  approach  to 
determine  the  associated  level  of  risk  with  each  proposal.  Performance  risk  is  the  analysis 
of  each  contractor’s  ability  to  meet  the  requirements  of  the  contract  based  upon  the 
contractor’s  capabilities  and  evidenced  by  previous  jobs  of  a  similar  scope  (Wright, 
2007).  The  criteria  evaluated  are  unique  to  each  procurement,  and  it  is  the  responsibility 
of  agency  acquisition  officials  to  ensure  the  source  selection  criteria  chosen  provide  a 
shelter  for  the  government  from  unnecessary  or  unforeseen  risks. 

As  mentioned  previously,  FAR  15.304  requires  that  price  or  cost  is  an  evaluation 
factor  in  all  procurements,  regardless  of  the  source  selection  strategy  chosen.  In 
evaluating  cost  or  price,  FAR  15.305  stipulates  the  following: 

Normally,  competition  establishes  price  reasonableness.  Therefore,  when 
contracting  on  a  firm-fixed-price  or  fixed-price  with  economic  price 
adjustment  basis,  comparison  of  the  proposed  prices  will  usually  satisfy 
the  requirement  to  perform  a  price  analysis,  and  a  cost  analysis  need  not 
be  performed.  In  limited  situations,  a  cost  analysis  may  be  appropriate  to 
establish  reasonableness  of  the  otherwise  successful  offeror’s  price.  When 
contracting  on  a  cost-reimbursement  basis,  evaluations  shall  include  a  cost 
realism  analysis  to  determine  what  the  Government  should  realistically 
expect  to  pay  for  the  proposed  effort,  the  offeror’s  understanding  of  the 
work,  and  the  offeror’s  ability  to  perform  the  contract. 
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As  directed  by  the  FAR,  each  proposal’s  price  is  evaluated  based  on  realism  and 
competitiveness.  Price  realism  determines  if  a  proposal  is  priced  too  low  to  realistically 
achieve  the  requirements  of  the  contract.  Cost  competitiveness  determines  if  the  proposed 
costs  are  unnecessarily  high.  Cost  competitiveness  is  evaluated  simply  by  looking  at  each 
proposed  price  against  other  competitive  proposals,  comparable  commercial  options,  and 
other  pricing  information  (Garrett,  2007). 

Price  analysis,  as  FAR  Part  15  suggests,  is  fairly  straightforward,  especially  when 
multiple  competitive  offers  are  received.  Price  analysis  can  also  be  conducted  using 
historical  prices  from  previous  procurements,  parametric  data,  published  prices, 
independent  government  cost  estimates,  and  market  research  (FAR,  2016). 

Cost  realism  analysis  is  much  more  in  depth  than  price  analysis  and  is  performed 
with  the  goal  of  determining  the  probable  cost  for  each  offeror  based  upon  each  offerors’ 
proposed  cost  in  a  cost  type  contract.  FAR  15.404  further  describes  cost  realism  analysis 
as 


the  process  of  independently  reviewing  and  evaluating  specific  elements 
of  each  offeror’s  proposed  cost  estimate  to  determine  whether  the 
estimated  proposed  cost  elements  are  realistic  for  the  work  to  be 
performed;  reflect  a  clear  understanding  of  the  requirements;  and  are 
consistent  with  the  unique  methods  of  performance  and  materials 
described  in  the  offeror’s  technical  proposal. 

FAR  15.404  goes  on  to  state  that  cost  realism  analysis  can  be  used  in  certain 
circumstances  to  evaluate  fixed  price  contracts  as  well.  Nash,  Cibinic,  and  O’Brien 
(1999)  clarify,  though,  that 

although  the  FAR  refers  to  this  process  as  cost  realism  analysis,  greater 
clarity  is  achieved  by  calling  it  “price  realism  analysis.”  This  signifies  that 
such  analysis  cannot  be  used  to  adjust  the  offered  prices  but  may  only  be 
used  to  make  a  responsibility  determination,  a  performance  risk 
assessment,  or  an  analysis  of  whether  the  offeror  understands  the  work.  (p. 

589) 

The  ultimate  goal  of  these  proposal  analysis  techniques  is  to  ensure  the  government 
receives  a  fair  and  reasonable  price.  The  source  selection  process  demands  extreme 
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caution  and  due  diligence  as  the  success  or  failure  of  the  acquisition,  once  awarded,  is 
largely  dependent  upon  the  contractor  selected  (Garrett,  2007). 

b.  Negotiation 

FAR  Part  15  states  that  “any  contract  awarded  using  other  than  sealed  bidding 
procedures  is  a  negotiated  contract.”  Contracting  by  negotiation  offers  some  flexibility  to 
the  award  phase,  allowing  contractors  to  revise  their  proposals  prior  to  award  and  assist 
in  the  formulation  of  the  terms  and  conditions  to  be  placed  in  the  contract  (Nash  et  al., 
2007).  Garrett  (2007)  explains  that 

the  ideal  is  to  develop  a  set  of  shared  expectations  and  understandings . 

The  buyer  and  seller  must  develop  and  agree  to  contract  terms  and 

conditions  that  are  designed  to  express  their  mutual  expectations  about 

performance  and  that  reflect  the  uncertainties  and  risks  of  performance,  (p. 

26) 

A  failure  to  communicate  during  this  step  can  bring  extreme  tension  to  the  business 
relationship  during  execution  of  the  contract  and  result  in  cost  and  schedule  growth 
(Garrett,  2007). 

3.  Post-Award  Phase 

After  the  contract  is  awarded,  the  contract  management  process  focuses  on 
monitoring  and  evaluating  the  contractor’s  performance  to  ensure  it  is  within  the  agreed 
upon  technical,  cost,  and  schedule  constraints,  and  on  executing  a  successful  closeout  or 
termination  of  the  contract.  Wright  (2007)  explains  that  “the  degree  of  monitoring  and 
surveillance  of  risk  areas  will  vary  depending  upon  the  credentials,  past  performance,  and 
experience  of  the  selected  proponent”  (p.  11).  For  both  the  government  and  the 
contractor,  the  post-award  phase  represents  the  “fulfillment  of  the  contractual  obligations 
by  all  parties  to  the  contract”  (Garrett,  2007,  p.  162  ). 

a.  Contract  Administration 

The  primary  purpose  of  contract  administration  is  ensuring  that  both  the 
government  and  the  contractor  are  in  “compliance  with  the  terms  and  conditions  of  the 
contract”  (Garrett,  2007,  p.  182).  Nash  et  al.  (2007)  list  some  of  the  tasks  involved  in  this 
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process:  “monitoring  contractor  progress,  reviewing  invoices,  processing  payments,  and 
inspecting  deliverables”  (p.  129).  Another  key  aspect  of  the  contract  administration 
process  is  managing  the  unexpected  (Nash  et  al.,  2007).  The  complex  nature  of  certain 
contracts  can  result  in  a  misunderstanding  of  the  requirements,  contractual  disputes,  or  in 
some  cases,  the  government’s  needs  evolving.  When  this  happens,  change  requests  or 
modifications  to  the  contract  are  both  common  and  necessary.  Changes  to  the  contract 
can  and  often  do  cause  increases  in  price  and  schedule.  Garrett  (2007)  recommends  that 
when  making  changes  to  the  contract,  both  parties  be  involved  in  estimating  the  impacts 
of  the  changes  on  cost  and  schedule.  It  saves  time  and  helps  ensure  both  parties 
understand  what  is  being  changed  and  the  impact  of  that  change.  In  some  cases,  the 
contractor  is  permitted  to  proceed  with  the  changes  prior  to  negotiation,  but  must  adhere 
to  a  cost  and  schedule  ceiling  set  by  the  government.  With  change  comes  risk  that  must 
be  managed.  Garrett  (2007)  states, 

Managing  change  means  ensuring  that  changes  are  authorized,  their  effect 
is  estimated  and  provided  for,  they  are  promptly  identified,  the  other  party 
is  properly  notified,  compliance  and  impact  are  reported,  compensation  is 
provided,  and  the  entire  transaction  is  properly  documented,  (p.  178) 

b.  Contract  Termination  or  Closeout 

Contract  closeout  involves  closing  out  the  contract,  whether  it  is  due  to  the 
successful  completion  of  the  contract  or  the  termination  for  other  reasons.  Garrett  (2007) 
asserts  that  “a  contract  can  end  in  one  of  three  ways:  successful  performance,  mutual 
agreement,  or  breach  of  contract”  (p.  185).  When  both  parties  meet  the  requirements  set 
forth  in  the  contract,  a  contract  is  closed  out.  Closing  out  a  contract  requires  the 
government  to  certify  that  the  contractor  has  completed  the  requirements  set  forth  in  the 
contract  and  the  contractor  to  certify  that  it  has  been  compensated  as  agreed  upon  in  the 
contract. 

There  are  two  types  of  terminations  as  defined  by  FAR  Part  49,  termination  for 
the  convenience  of  the  government  or  default  of  the  contractor.  Garrett  (2007)  provides  a 
third  option  for  termination:  mutual  agreement.  Termination  by  mutual  agreement  occurs 
when  both  parties  agree  that  they  no  longer  want  to  be  bound  by  the  terms  set  forth  in  the 
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contract  (Garrett,  2007).  FAR  2.101  defines  termination  for  cause  or  default  as  “the 
exercise  of  the  Government’s  right  to  completely  or  partially  terminate  a  contract  because 
of  the  contractor’s  actual  or  anticipated  failure  to  perform  its  contractual  obligations.” 
FAR  2.101  defines  termination  for  convenience  as  “the  exercise  of  the  Government’s 
right  to  completely  or  partially  terminate  performance  of  work  under  a  contract  when  it  is 
in  the  Government’s  interest.”  Garrett  (2007)  highlights  that  this  unilateral  right  of  the 
government  to  terminate  for  convenience  has  been  the  “subject  of  many  U.S.  court  and 
legal  decisions”  (p.  191). 

This  section  on  the  post-award  phase  focused  on  how  the  government  monitors 
and  evaluates  contractor  performance  throughout  the  execution  of  the  contract  and 
discussed  how  the  contract  is  closed  out  or  terminated.  The  close-out  or  termination  of 
the  contract  completes  the  contract  management  process.  The  next  section  includes  a 
detailed  discussion  of  the  source  selection  strategy,  which  is  developed  during  the 
procurement  planning  stage. 

C.  SOURCE  SELECTION  STRATEGY 

The  DOD  source  selection  procedure  provides  the  framework  for  developing  the 
source  selection  strategy  and  is  required  for  all  competitive  contracts  by  negotiation 
(Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
[OUSD(AT&L)],  2011).  The  best  strategy  can  be  derived  from  the  evaluation  of  where 
the  acquisition  falls  within  the  best  value  continuum.  FAR  15.101  states, 

An  agency  can  obtain  best  value  in  negotiated  acquisitions  by  using  any 
one  or  a  combination  of  source  selection  approaches.  In  different  types  of 
acquisitions,  the  relative  importance  of  cost  or  price  may  vary.  For 
example,  in  acquisitions  where  the  requirement  is  clearly  definable  and  the 
risk  of  unsuccessful  contract  performance  is  minimal,  cost  or  price  may 
play  a  dominant  role  in  source  selection.  The  less  definitive  the 
requirement,  the  more  development  work  required,  or  the  greater  the 
performance  risk,  the  more  technical  or  past  performance  considerations 
may  play  a  dominant  role  in  source  selection. 

The  government  has  two  strategies  it  can  choose  between  when  developing  the  source 
selection  strategy  for  a  competitive,  negotiated  contract.  As  FAR  15.101  describes,  some 
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acquisitions  have  requirements  that  are  “clearly  definable,”  and  the  risk  of  the  contractor 
failing  to  perform  is  minimal.  In  this  case,  the  lowest  price  technically  acceptable  (LPTA) 
source  selection  strategy  may  be  appropriate.  On  the  other  hand,  if  the  requirement  is 
“less  definitive,  more  development  work  is  required,  or  there  is  greater  performance  risk” 
(FAR  15.101),  the  tradeoff  strategy  may  be  more  appropriate.  FAR  15.101  further 
explains  that  the  “tradeoff  process  is  appropriate  when  it  may  be  in  the  best  interest  of  the 
Government  to  consider  award  to  other  than  the  lowest  priced  offeror  or  other  than  the 
highest  technically  rated  offeror  (HTRO).”  The  best  value  continuum  can  be  thought  of 
as  the  range  between  cost  or  price  evaluation  factors  and  the  relative  importance  of  non¬ 
cost  or  price  factors.  Figure  3  provides  a  visualization  of  the  best  value  continuum. 
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Figure  3.  Best  Value  Continuum.  Adapted  from  Naval  Air  Warfare  Center 

(NAWC;  2015). 


The  ultimate  goal  of  the  government  is  to  obtain  the  best  value.  FAR  2.101 


defines  best  value  as  “the  expected  outcome  of  an  acquisition  that,  in  the  Government’s 

18 


estimation,  provides  the  greatest  overall  benefit  in  response  to  the  requirement.”  In 
developing  the  source  selection  strategy,  the  first  step  is  determining  the  relative 
importance  of  cost  and  price  to  other  evaluation  factors,  and  identifying  what  those 
evaluation  factors,  other  than  cost  and  price,  should  be.  This  will  determine  whether 
LPTA  or  tradeoff  should  be  used  to  achieve  the  desired  best  value  to  the  government 
(Rumbaugh,  2010). 

1.  Lowest  Price  Technically  Acceptable 

FAR  15.101  states  that  “the  lowest  price  technically  acceptable  source  selection 
process  is  appropriate  when  best  value  is  expected  to  result  from  selection  of  the 
technically  acceptable  proposal  with  the  lowest  evaluated  price.”  Price  and  cost  are  not 
the  only  evaluation  factors  considered  in  an  LPTA  decision,  however.  The  technical 
acceptability  of  a  proposal  may  contain  other  non-cost  factors  such  as  past  performance 
or  management  capability,  which  are  evaluated  on  a  pass/fail  basis.  These  non-cost 
factors  must  meet  the  government’s  level  of  acceptability  for  the  offer  to  then  be  ranked 
according  to  price.  Once  an  acceptable  range  of  offers  is  established  based  upon  technical 
acceptance,  the  government  must  select  the  lowest  priced  offer  (Rumbaugh,  2010). 

2.  Tradeoff 

FAR  15.101  states,  “A  tradeoff  process  is  appropriate  when  it  may  be  in  the  best 
interest  of  the  Government  to  consider  award  to  other  than  the  lowest  priced  offeror  or 
other  than  the  highest  technically  rated  offeror.”  The  risk  associated  with  certain 
acquisitions  may  merit  the  use  of  the  tradeoff  process.  If  the  government  requirements  are 
difficult  to  define  or  there  are  multiple  possible  solutions  to  the  government  requirement, 
tradeoff  may  be  the  best  source  selection  strategy  (Rumbaugh,  2010).  Non-cost  factors 
can  be  given  more  weight,  less  weight,  or  equal  weight  to  cost  and  price  at  the 
government’s  discretion.  These  non-cost  factors  can  include  technical  criteria, 
management  criteria,  past  performance,  and  other  qualitative  or  quantitative  criteria 
(Garrett,  2007).  These  weights  must  all  be  stated  within  the  solicitation  notifying 
potential  offerors  how  offers  will  be  evaluated.  FAR  15.101  explains  that,  in  making  the 
selection,  “the  perceived  benefits  of  the  higher  priced  proposal  shall  merit  the  additional 
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cost,  and  the  rationale  for  tradeoffs  must  be  documented.”  Just  as  with  LPTA,  the 
ultimate  goal  of  the  tradeoff  strategy  is  to  obtain  the  best  value  for  the  government. 

Given  the  immense  importance  of  the  source  selection  strategy,  which  is 
developed  during  the  procurement  planning  step,  it  is  reasonable  to  conclude  that  the 
most  significant  step  in  the  contract  management  process  is  procurement  planning. 
Defining  the  requirement  and  conducting  market  research  sets  the  course  for  the  rest  of 
the  contract  management  process.  Given  the  significance  of  the  source  selection  strategy 
to  the  contract  management  process,  the  following  sections  provide  insight  from 
investigative  reports,  industry  perspective,  and  ongoing  debates  regarding  current  DOD 
practices  and  priorities  when  developing  the  source  selection  strategy. 

D.  INVESTIGATIVE  REPORTS 

Due  to  recent  budget  constraints,  the  DOD  faces  increased  scrutiny  from 
policymakers  over  its  contract  management.  Since  1992,  the  GAO,  the  investigative  arm 
of  Congress,  maintains  DOD  contract  management  in  its  high-risk  category  because  the 
DOD  continues  to  encounter  challenges  in  providing  relevant  skills  and  capabilities  to  its 
contracting  professionals,  implementing  sound  techniques  and  approaches  to  awarding 
contracts,  and  properly  managing  its  acquisition  of  services  and  contingency  contracting 
(GAO,  2015).  The  GAO,  as  well  as  congressional  policymakers,  recently  lauded  the 
DOD’s  efforts  to  tackle  its  contract  management  issues  (H.R.  Rep.  113-102,  2013).  The 
DOD’s  incremental  improvements  are  reflected  in  the  GAO’s  2015  High-Risk  Series 
report  to  Congress.  The  improvements  are  due  in  part  to  the  BBP  initiatives  created 
specifically  to  address  contract  management  issues  by  promoting  better  efficiencies, 
controlling  costs,  cutting  burdensome  bureaucratic  processes,  and  training  its  acquisition 
workforce  (GAO,  2014). 

Still,  Congress  is  concerned  that  the  DOD’s  efforts  to  achieve  the  goals  of  the 
BBP  motivates  contracting  professionals  to  employ  LPTA  as  the  default  source  selection 
strategy  and  inappropriately  award  LPTAs  when  a  tradeoff  approach  is  more  suited. 
Therefore,  as  part  of  the  National  Defense  Authorization  Act  of  2014,  the  House  of 
Representatives  and  Senate  committees  required  the  DOD  to  report  on  its  use  of  the  best 
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value  processes  in  hopes  of  ensuring  that  DOD  contracting  officials  correctly  determine 
the  complexity  of  requirements,  properly  assign  performance  risk,  and  carefully  assess 
whether  failure  to  perform  has  significant  implications  (H.R.  Rep.  113-102,  2013;  GAO, 
2014).  Additionally,  a  House  of  Representatives  report  encouraged  the  DOD  to  obtain 
the  views  of  the  defense  industry  to  determine  how  source  selection  strategies  affect  the 
industry’s  business  decisions,  since  part  of  the  design  of  the  best  value  continuum  is  to 
foster  competition  and  innovation  within  the  defense  industry  (H.R.  Rep.  113-102, 

2013) . 

In  2014,  the  GAO  conducted  a  congressionally  mandated  review  to  determine  the 
scope  of  the  DOD’s  use  of  best  value  strategies.  From  its  review  of  2,851  contracts 
awarded  in  fiscal  year  (FY)  2013,  the  GAO  (2014)  determined  that  93%  of  the  DOD’s 
competitively  awarded  contracts  utilized  best  value  processes.  Sealed  bidding  accounted 
for  the  remaining  6%  of  contracts  awarded.  For  contracts  over  $25  million,  the  DOD  used 
tradeoff  58%  of  the  time  compared  to  LPTA  36%  of  the  time  (GAO,  2014).  For  contracts 
over  $1  million  but  less  than  $25  million,  DOD  use  of  LPTA  and  tradeoff  was  nearly 
equal.  The  DOD’s  use  of  LPTA  rose  by  10%  from  a  previous  GAO  review  of  contracts 
awarded  in  FY2009  (GAO,  2010).  The  DOD  employed  tradeoff  strategies  to  acquire 
construction,  maintenance,  and  other  support  services  regardless  of  obligation,  but  for 
contracts  greater  than  $25  million,  LPTA  strategies  were  mostly  used  to  acquire  products. 
However,  for  contracts  more  than  $1  million  but  less  than  $25  million,  contracting 
professionals  employed  LPTA  to  acquire  a  combination  of  products  and  services  (GAO, 

2014) . 

DOD  contracting  professionals’  desire  to  meet  BBP  initiatives  may  drive  them  to 
employ  LPTA  more  often.  The  GAO  interviewed  several  officials  from  different 
commands  across  all  services.  These  senior  officials  indicated  that  declining  budgets  and 
efforts  to  simplify  requirements  motivate  commands  to  utilize  LPTA  more  often  than 
tradeoff  (GAO,  2014). 

Contracting  professionals  at  NAVSEA  interviewed  by  the  GAO  say  senior 

command  officials  are  placing  more  scrutiny  on  tradeoff  procedures  and  instituting  a 

“cultural  shift”  that  normalizes  LPTA  as  the  preferred  source  selection  strategy.  As 
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evidenced  by  a  NAVSEA  memo  to  its  contracting  operations,  a  senior  level  acquisition 
executive  must  review  the  details  of  a  potential  acquisition  if  program  managers 
determine  that  a  non-cost  factor  is  more  important  than  price.  Further,  officials 
throughout  all  DOD  services  and  agencies  report  that  commands  are  shifting  focus  to 
utilize  LPTA  more  often  (GAO,  2014). 

The  previous  discussion  addressed  investigative  reports  that  indicate  the  DOD’s 
increased  frequency  of  employing  LPTA.  The  next  section  discusses  defense  industry 
concerns  that  the  increased  use  of  LPTA  reduces  product  and  service  innovation, 
constraining  companies  to  cut  costs  to  provide  only  the  most  marginally  acceptable 
product  or  service  to  win  a  contract. 

E.  INDUSTRY  PERSPECTIVE 

Recently,  the  president  of  a  satellite  service  provider  suggested  that  the  more 
frequent  use  of  LPTA  curbs  defense  industry  innovation  and  capabilities,  a  sentiment 
shared  by  many  in  the  defense  industry.  He  continued,  “In  my  opinion,  I  think  the 
warfighter  is  getting  a  raw  deal”  (Magnuson,  2014,  p.  1).  The  trend  to  use  LPTA  more 
often  is  drawing  sharp  criticism  from  operators  in  the  defense  industry  (Magnuson, 
2014).  Defense  industry  experts  contend  that  overusing  LPTA  will  pressure  industry  to 
reduce  innovation  and  compel  it  to  leave  the  defense  marketplace  altogether. 
Furthermore,  industry  experts  argue  that  the  FAR  should  be  updated  to  narrow  the  range 
of  types  of  solicitations  LPTA  could  be  used  for  (Goodman,  2015). 

Will  Goodman  (2015),  assistant  vice  president  for  policy  at  the  National  Defense 
Industrial  Association,  argues  that  in  a  marketplace  in  which  tradeoff  is  the  primary 
source  selection  strategy,  companies  will  respond  by  attempting  to  produce  the  best 
product  at  the  most  competitive  price.  However,  in  an  LPTA-dominant  industry, 
companies  are  incentivized  to  reduce  their  costs  as  low  as  possible  to  a  point  where  they 
provides  a  service  or  product  that  marginally  meets  the  technically  acceptable 
requirement.  Industry  will  reduce  product  innovation  and,  in  an  ironic  twist,  innovate  in 
ways  that  reduce  costs  and  product  quality  to  meet  the  lowest  price  target.  By  employing 
only  an  LPTA  solution,  Goodman  argues  the  DOD  is  telling  contractors  to  “make  [a] 
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product  as  cheaply  as  you  can  ...  as  long  as  it  meets  minimum  requirements.  If  you 
refuse,  we  will  punish  you  by  awarding  the  contract  to  someone  else”  (Goodman,  2015, 
p.  18).  As  discussed  in  the  literature  review,  Goodman’s  argument  can  be  countered  by 
applying  price  realism  and  cost  competitiveness.  Price  realism  is  used  to  determine  if  a 
proposed  price  is  too  low  to  realistically  achieve  the  requirements  of  the  contract,  and 
cost  competiveness  is  used  to  determine  if  the  price  is  too  high  compared  to  the  level  of 
effort  to  achieve  the  requirement  (Garrett,  2007). 

Goodman  (2015)  concludes  by  arguing  that  the  FAR  is  incomplete  and  does  not 
adequately  address  the  economic  and  future  capability  consequences  from  employing 
LPTA  strategies.  Goodman  contends  that  the  FAR  should  explicitly  state  that  LPTA  is 
best  used  for  commodities  and  commoditized  services,  and  acquisition  professionals 
should  consider  the  long-term  implications  of  a  capability  area  required  by  the 
solicitation  (Goodman,  2015). 

We  next  discuss  the  ongoing  debate  regarding  source  selection  strategies  and  how 
decision  makers  should  first  assess  several  multifunctional  indicators  to  determine  a 
contract’s  complexity  and  risk  and  use  that  determination  to  attain  the  best  value  in  a 
source  selection. 

F.  ONGOING  DEBATE 

Previously,  we  discussed  GAO  reports  revealing  the  DOD’s  current  inclination  to 
award  LPTA  contracts  more  frequently  compared  to  previous  years  and  defense  industry 
perspectives  that  the  increased  use  of  LPTA  is  stifling  competition  and  innovation,  and 
driving  down  industry’s  profit.  In  response  to  industry’s  concerns,  two  Defense 
Acquisition  University  (DAU)  faculty  members,  Brian  Schultz  and  David  Dotson  (2015), 
propose  a  more  holistic  approach  to  assist  contracting  officers  in  selecting  the  best  source 
selection  strategy. 

Schultz  and  Dotson  (2015)  argue  that  Goodman’s  (2015)  proposed  changes  to  the 
FAR  are  not  feasible  since  program-level  contracting  officers  should  not  be  expected 
to — nor  have  the  capability  to — determine  the  economic  consequences  of  solicitations 
over  an  extended  period  of  time.  Schultz  and  Dotson  propose  a  different  methodology 
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that  enlists  program  decision  makers,  with  support  of  the  contracting  officer,  to  assess 
multifunctional  areas  beyond  just  performance  requirements  to  help  program  managers 
settle  on  the  best  value  in  a  source  selection.  The  multifunctional  areas  program  decision 
makers  should  evaluate  are  the  contractor’s  organizational  landscape,  the  requirement’s 
mission  and  operational  interfaces,  industrial  capabilities  to  deliver  a  requirement, 
contract  deliverables,  and  risk  and  opportunity  management  (Schultz  &  Dotson,  2015). 

First,  Schultz  and  Dotson  (2015)  maintain  that  organizations  that  are  adequately 
staffed,  experience  low  management  turnover,  and  align  resources  to  their  strategy  and 
goals  will  be  more  suited  to  administer  and  manage  complex  tradeoff  contracts. 
Contracting  professionals  should  determine  if  organizations  possess  these  qualities,  since 
well-managed  organizations  plan,  resource,  and  execute  highly  complex  acquisition 
programs  more  adroitly  than  poorly  managed  organizations  (Schultz  &  Dotson,  2015). 

Next,  program  decision  makers  should  determine  whether  a  requirement  needs  to 
be  integrated  into  the  overall  operational  environment.  In  other  words,  a  requirement  may 
need  to  exist,  operate,  and  interface  with  multiple  systems  within  an  environment. 
Program  decision  makers  should  consider  the  risks  and  complexities  of  an  acquisition  if  a 
rapidly  evolving  technical  requirement  must  operate  in  an  operational  environment 
(Schultz  &  Dotson,  2015). 

Third,  program  decision  makers  should  determine  industry’s  ability  to  produce  a 
requirement.  They  should  study  a  contractor’s  past  performance  records  to  include  a 
review  of  that  contractor’s  supplier  performance.  Poor  first  and  second  tier  suppliers  can 
lead  to  schedule  delays  and  quality  issues.  This  information  can  provide  relevant  insight 
into  the  potential  risks  and  complexities  of  an  acquisition.  Also,  program  decision  makers 
should  conduct  in-depth  market  research  to  gather  pertinent  information  regarding  the 
character  of  the  acquisition  and  the  industry  that  will  produce  the  acquisition.  For 
example,  program  decision  makers  may  determine  that  a  requirement’s  design  is  highly 
complex,  yet  ultimately  conclude  that  the  acquisition  is  low  risk  because  companies  that 
operate  within  the  industry  are  adept  at  making  the  system  (Schultz  &  Dotson,  2015). 
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Fourth,  program  decision  makers  should  clarify  exactly  what  they  need,  how 
much  of  it  they  want,  and  how  much  they  are  willing  to  pay  for  it.  They  should  spell  out 
exactly  what  a  successful  contractual  performance  would  entail.  Additionally,  program 
decision  makers  should  determine  the  monetized  value  of  performance  above  the 
minimum  threshold  requirement,  a  key  process  that  Frank  Kendall  writes  about  to 
determine  an  appropriate  source  selection  strategy  (Schultz  &  Dotson,  2015). 

Finally,  after  reviewing  the  first  four  functional  areas  described  by  Schultz  and 
Dotson  (2015) — organizational  structure,  operational  interfaces,  industry  capabilities,  and 
contract  deliverables — program  decision  makers  should  pinpoint  and  calculate  risk  areas 
and  ways  to  mitigate  these  risks.  Putting  it  all  together,  program  decision  makers  should 
generate  an  assessment  that  deems  the  acquisition  as  low,  moderate,  or  high  complexity 
and  risk,  based  on  the  multifunctional  assessment.  Schultz  and  Dotson  (2015)  then  apply 
the  assessment  to  a  color  schematic  “integrated  assessment  rating  scale”  where  green  is 
portrayed  as  low  complexity  and  risk,  yellow  as  moderate  complexity  and  risk,  and  red  as 
high  complexity  and  risk  (p.  7).  Program  decision  makers  should  view  an  acquisition 
assessed  in  the  red  category  as  a  candidate  for  a  tradeoff  source  selection  strategy, 
whereas  acquisitions  that  fall  towards  the  green  side  of  the  scale  may  be  awarded  via  the 
LPTA  strategy  (Schultz  &  Dotson,  2015). 

G.  SUMMARY 

The  preceding  literature  review  covered  the  statutes  and  regulations  applicable  to 
the  contract  management  process,  the  steps  of  the  contract  management  process,  and 
source  selection  strategies.  The  literature  review  also  identified  GAO  reports  that  show 
the  DOD’s  inclination  to  employ  LPTA  more  often  in  recent  years,  the  defense  industry’s 
response  to  the  increased  prevalence  of  LPTA,  as  well  as  the  ongoing  debate  regarding 
source  selection  strategies.  The  next  chapter  discusses  an  overview  of  DOD  acquisition 
organizations.  Navy  acquisition  organizations,  and  NAVSEA. 
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III.  DOD  ACQUISITION  ORGANIZATION 


This  chapter  provides  an  overview  of  the  DOD  acquisition  organizations,  Navy 
acquisition  organizations,  and  Naval  Sea  Systems  Command  (NAVSEA).  The  research 
provides  information  on  each  organization’s  responsibilities,  structure,  mission,  and 
function.  It  also  highlights  the  layers  of  bureaucracy  and  oversight  within  the  defense 
acquisition  system. 

A.  DOD  ACQUISITION  STRUCTURE 

The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
(USD[AT&LJ)  is  “the  principal  assistant  and  advisor  to  the  Secretary  of  Defense  and 
Deputy  Secretary  of  Defense  for  all  matters  concerning  acquisition,  technology,  and 
logistics”  (OUSD[AT&L],  2016,  para.  1).  As  outlined  in  10  U.S.C.  §  133  (2011),  “the 
primary  responsibilities  of  the  USD(AT&L)  include:  supervising  acquisition  in  the 
Department  of  Defense;  establishing  policies  for  acquisition;  logistics;  and,  maintaining 
and  sustaining  of  all  elements  within  the  Department  of  Defense.”  Additionally,  the 
USD(AT&L)  is  responsible  for  developing  policy  for  the  maintenance  of  the  defense 
industrial  base  and  has  the  authority  to  direct  the  service  secretaries  with  regard  to 
matters  under  which  the  under  secretary  has  responsibility  (10  U.S.C.  §  133,  2011).  The 
USD(AT&L)  has  oversight  responsibility  for  space  and  intelligence  programs,  missile 
defense  programs,  operational  energy  programs  and  plans,  military  installations  and  their 
environment,  and  major  weapon  systems  (OUSD[AT&L],  2016a).  The  USD(AT&L)  also 
serves  as  the  defense  acquisition  executive  (DAE)  for  the  Milestone  Decision  Authority 
(MDA)  on  major  weapon  system  acquisition.  Figure  4  illustrates  the  hierarchical 
structure  of  the  OUSD(AT&L). 
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Figure  4.  OUSD(AT&L)  Organizational  Structure.  Adapted  from 

OUSD(AT&L)  (2016b). 


Each  military  component  has  a  service  acquisition  executive  (SAE)  who  reports 
to  the  DAE.  Within  the  military  departments,  the  officials  delegated  as  SAEs  are  as 
follows:  the  assistant  secretary  of  the  Navy  for  research,  development,  and  acquisition 
(ASN[RD&A]);  the  assistant  secretary  of  the  Army  for  acquisition,  logistics,  and 
technology  (ASA[AL&T]);  and  the  assistant  secretary  of  the  Air  Force  for  acquisition 
(ASAF[A]).  In  turn,  each  SAE  is  responsible  for  developing  and  implementing 
procedures  within  the  respective  components  in  accordance  with  the  DOD  5000  Series 
Directives  (DOD,  2012). 

Within  the  Office  of  the  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (OSD[AT&L]),  the  Director  of  Defense  Procurement  and  Acquisition  Policy 
(DPAP)  develops  and  implements  policy  through  the  Defense  Federal  Acquisition 
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Regulation  Supplement  (DFARS);  procedures,  guidance,  and  information  (PGI)  updates; 
and  DOD  Directive  5000.01  and  DOD  Directive  5000.02.  The  OSD(AT&L)’s  mission  is 
to  “provide  innovative  acquisition  and  procurement  policy  that  will  effectively  deliver 
services  and  equipment  to  the  warfighter  while  being  good  stewards  of  the  taxpayers’ 
dollar”  (DPAP,  n.d.-c,  para.  1). 

There  are  seven  directorates  within  DPAP  that  support  this  mission,  as  outlined  in 
Figure  5.  The  operations  directorate  provides  oversight  to  the  other  six  directorates  and 
manages  budget  planning,  travel,  human  resources,  legislative  concerns,  strategic 
planning,  and  workforce  management  for  the  contracting  community.  The  Defense 
Acquisition  Regulations  System  (DARS)  directorate  develops  and  maintains  the  DFARS, 
publishes  PGI  updates,  leads  the  Defense  Acquisition  Regulations  (DAR)  Council,  and 
issues  and  maintains  class  deviations.  The  Contract  Policy  and  International  Contracting 
(CPIC)  directorate  is  responsible  for  “developing  new,  innovative  acquisition  policies 
and  improving  existing  DOD  acquisition  policies”  (DPAP,  n.d.-c,  para.  1).  The 
contingency  contracting  directorate  is  responsible  for  policy  and  resources  when 
contracting  is  performed  in  a  contingency  environment,  and  the  program  acquisition 
directorate  is  responsible  for  the  development  and  supervision  of  procurement  strategies 
of  major  defense  acquisition  programs.  Lastly,  a  relatively  new  directorate  is  services 
acquisition  and  strategic  sourcing.  Established  in  2013,  this  directorate  is  responsible  for 
all  aspects  of  service  acquisition  oversight  to  include  “developing,  implementing, 
governing,  and  executing  the  acquisition  oversight  framework  of  services,  and  for  the 
championing  of  strategic  sourcing  policy  and  initiatives,  for  the  DOD”  (DPAP,  n.d.-a 
para.  1). 
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Figure  5.  DPAP  Organizational  Structure.  Adapted  from  DPAP  (n.d.-d). 


B.  NAVY  ACQUISITION  ORGANIZATION 

The  assistant  secretary  of  the  Navy  for  research,  development  and  acquisition 
(ASN[RD&A])  serves  as  the  Navy  acquisition  executive.  The  mission  of  the  office  is  to 
develop,  acquire,  and  deliver  state-of-the-art  platforms,  systems,  and  weapons  that  will 
give  Sailors  and  Marine  Corps  personnel  an  asymmetrical  advantage  over  the  nation’s 
adversaries  (ASN[RD&A],  n.d.-b).  The  ASN(RD&A)  is  accountable  and  responsible  for 
all  acquisitions  functions,  programs,  and  policy  under  the  Department  of  the  Navy 
umbrella.  The  Office  of  the  ASN(RD&A)  consists  of  an  immediate  staff,  direct  reporting 
program  managers  (DRPMs),  program  executive  officers  (PEOs),  and  program  managers 
(PMs). 

As  a  result  of  the  1986  Packard  Commission  report,  NAVSEA,  Naval  Air 
Systems  Command  (NAVAIR)  and  Space  and  Naval  Warfare  Systems  Command 
(SPAWAR),  were  the  first  Navy  systems  commands  (SYSCOMs)  to  implement  a  PEO 
structure  within  each  of  their  organizations.  The  PEO  structure  creates  clear  lines  of 
accountability  and  responsibility,  as  the  PEO  is  the  main  stakeholder  responsible  for  cost, 
schedule,  and  performance  in  major  acquisition  programs  or  portfolios  of  programs 
(DOD,  2007).  DRPMs  are  also  the  main  stakeholder  and  are  assigned  to  challenging 
acquisition  programs  for  temporary  periods  of  time  to  resolve  critical  acquisition  issues. 
There  are  15  standalone  Navy  PEOs  and  DRPMs: 

•  PEO  Joint  Strike  Fighter 

•  PEO  Ships 

•  PEO  Submarines 
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PEO  Aircraft  Carriers 


•  PEO  Tactical  Air  Program 

•  PEO  Air  Anti-Submarine  Warfare 

•  PEO  Assault  and  Special  Mission  Programs 

•  PEO  Strike  Weapons  and  Unmanned  Aviation 

•  PEO  Space  Systems 

•  PEO  Littoral  and  Mine  Warfare 

•  PEO  Integrated  Warfare  Systems 

•  PEO  Enterprise  Information  Systems 

•  PEO  Command,  Control,  Communications,  Computers,  and  Intelligence 
(C4I) 

•  PEO  Marine  Land  Systems 

•  DRPM  Strategic  Systems  Programs  (ASN[RD&A],  n.d.  a) 

Figure  6  illustrates  the  Navy’s  acquisition  reporting  structure  for  PMs,  PEOs, 
DRPMs,  and  SYSCOMs.  PMs  report  through  their  appropriate  PEO  or  SYSCOM  and 
through  them  to  the  ASN(RDA).  SYSCOMs  also  report  to  the  Chief  of  Naval  Operation 
(CNO)  for  administrative  and  inter-service  support.  DRPMs  report  directly  to  the 
ASN(RDA;  DOD,  2007). 
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Figure  6.  Reporting  Chain  for  Navy  Acquisition. 

Adapted  from  DOD  (2007). 


C.  NAVAL  SEA  SYSTEMS  COMMAND 

NAVSEA’s  mission  is  to  “build,  design,  deliver  and  maintain  ships  and  systems 
on  time  and  on  cost  for  the  United  States  Navy”  (NAVSEA,  2013,  p.  1).  Its  vision 
statement  is  to  be  “the  nation’s  team,  accountable  for  the  health  of  the  United  States 
Navy’s  fleet  of  ships  and  to  set  the  value-added  standard  for  acquisition,  engineering, 
business,  and  maintenance”  (NAVSEA,  2013,  p.  2).  NAVSEA  is  the  largest  SYSCOM  in 
the  Navy  in  terms  of  both  workforce  and  budget.  Its  yearly  budget  of  approximately  $30 
billion  represents  one-fourth  of  the  Navy’s  overall  budget  (NAVSEA,  2013).  Figure  7 
depicts  how  NAVSEA  and  the  other  four  SYSCOMs  align  with  other  naval  shore 
establishments. 
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Figure  7.  Navy  SYSCOMs  in  Relation  to  Other  Navy  Shore  Establishments. 

Adapted  from  United  States  Navy  (n.d.). 

Figure  8  depicts  NAVSEA’s  command  staff:  its  headquarter  directorates  and  its 
five  affiliated  PEOs  and  field  activities.  Together,  “they  design,  build,  procure,  and 
maintain  ships,  submarines,  and  combat  systems  that  meet  current  and  future  operational 
requirements”  (NAVSEA,  n.d.-apara.  1). 
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Figure  8.  NAVSEA  Corporate  Leadership  2016.  Source:  NAVSEA  (2016). 


NAVSEA  has  five  PEOs  who  are  responsible  for  the  development,  acquisition, 
and  life-cycle  management  of  their  assigned  programs.  For  planning  and  execution  of 
life-cycle  management  and  sustainment,  PEOs  report  to  the  NAVSEA  commander.  For 
acquisition-related  matters,  PEOs  report  to  the  ASN(RD&A).  The  five  affiliated 
NAVSEA  PEOs  and  a  brief  description  of  each  follows: 

•  PEO  Aircraft  Carriers  manages  all  aircraft  carrier  life-cycle  support, 
design,  construction,  and  delivery. 

•  PEO  Integrated  Warfare  Systems  manages  all  combat  systems  for  surface 
ships  and  submarines. 

•  PEO  Littoral  Combat  Ships  manages  design,  procurement,  fleet 
employment,  and  sustainment  for  littoral  combat  ships. 

•  PEO  Ships  manages  and  is  responsible  for  research,  development, 
acquisition,  and  life-cycle  support  for  all  non-nuclear  surface  ships. 


34 


•  PEO  Submarines  manages  the  design,  construction,  and  delivery  of  all 
submarines,  advanced  undersea  systems,  and  anti-submarine  systems 
(NAYSEA,  n.d.-c). 

Each  year,  NAVSEA’s  contracts  directorate  (SEA  02)  and  its  field  contracting 
offices  process  approximately  60,000  contracting  actions  for  modification,  new 
construction  of  ships  and  submarines,  weapon  systems,  maintenance,  repair,  and  services. 
SEA  02  is  headquartered  at  the  Naval  Shipyard  in  Washington,  DC.  It  is  broken  down 
into  five  divisions,  which  specialize  in  policy,  shipbuilding,  fleet  support,  surface 
systems,  and  undersea  systems,  respectively.  The  Shipbuilding  Contracts  Division 
supports  four  field  procurement  offices:  the  Supervisor  of  Shipbuilding  in  Newport 
News,  VA;  Bath,  ME;  Groton,  CT;  and  Gulf  Coast  Pascagoula,  MS.  The  Fleet  Support 
Contracts  Division  is  also  headquartered  at  the  Naval  Shipyard  and  supports  two  naval 
shipyards  located  in  Pearl  Harbor,  HI,  and  Kittery,  ME.  It  also  supports  four  regional 
maintenance  centers  located  in  San  Diego,  CA;  Bremerton,  WA;  Norfolk,  VA;  and 
Mayport,  FL.  The  Surface  Systems  Contracts  Division  supports  seven  warfare  centers 
located  in  Panama  City,  FL;  Indian  Head,  MD;  Crane,  IN;  Port  Hueneme,  CA;  Corona, 
CA;  Philadelphia,  PA;  and  Dahlgren,  VA.  SEA  02’s  Undersea  Contracts  Division 
supports  two  undersea  warfare  centers  located  in  Keyport,  WA,  and  Newport,  RI 
(NAVSEA,  n.d.-b).  Together,  they  provide  a  wide  variety  of  acquisition  and  contract 
support  to  efficiently  maintain  a  fleet  of  nearly  380  ships  and  submarines,  their  weapon 
systems,  and  supporting  infrastructure. 

D.  WHY  WAS  NAVSEA  CHOSEN  FOR  OUR  RESEARCH 

NAVSEA  was  selected  for  this  research  project  because  it  procures  complex  sea 
systems  that  are  unique  to  the  Navy,  but  does  so  through  the  same  contract  management 
processes  used  throughout  the  DOD  and  the  federal  government.  Additionally,  NAVSEA 
was  selected  for  this  research  because  it  is  the  largest  procurement  organization  in  the 
Navy  that  conducts  thousands  of  contracting  source  selections  each  year  that  will  assist  in 
answering  our  research  questions. 
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E.  SUMMARY 


Chapter  III  provided  an  overview  of  the  DOD’s  acquisition  mission, 
responsibilities,  and  organization.  It  also  focused  on  the  Navy’s  organization  of 
acquisition  activities  and  its  largest  SYSCOM,  NAVSEA.  Chapter  IV  provides  a 
description  of  the  data  collected,  data  analysis,  and  findings. 
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IV.  ANALYSIS 


In  this  chapter,  we  provide  an  overview  of  prior  research  teams  that  conducted 
similar  research,  and  we  discuss  the  research  methodology  we  used  to  answer  our 
research  questions.  Specifically,  we  address  the  variables  examined  and  their  associated 
descriptive  statistics,  the  analysis  method  we  used,  and  the  problems  associated  with  our 
data.  Finally,  we  present  the  answers  to  our  research  questions. 

A.  OVERVIEW  OF  PRIOR  RESEARCH 

Our  research  is  part  of  an  ongoing  research  initiative  that  is  attempting  to 
determine  if  a  relationship  exists  between  source  selection  strategies  and  performance 
outcomes.  Using  the  same  methodology  and  data  collection  guide,  but  utilizing  different 
streams  of  data,  our  research  attempts  to  answer  the  same  or  very  similar  research 
questions  proposed  by  the  following  Naval  Postgraduate  School  research  teams: 

•  Watson  (2015) 

•  Lamoureux,  Murrow,  &  Walls  (2015) 

•  Ban,  Barnes,  &  Comer  (2015) 

•  B  as  tola,  Findley,  &  Woodward  (2015) 

•  Hill,  Odom,  Osman,  &  Paul  (2016) 

Our  data  collection  guide  (Appendix)  was  an  Excel  spreadsheet  developed  by  our 
advisors  to  guide  our  research  efforts  using  a  systematic  and  standardized  approach. 

B.  VARIABLES  EXAMINED 

For  this  analysis,  PALT  and  CPARS  ratings  were  used  as  the  two  main 

dependent,  or  outcome,  variables  (DVs).  PALT  is  a  continuous  variable  that  is  calculated 

as  the  date  between  receipt  of  requisition  and  the  date  of  contract  award.  Contracting 

professionals  assign  CPARS  ratings  to  a  completed  contract  to  measure  the  performance 

of  the  contract.  Each  contract’s  CPARS  rating  was  based  on  the  following  variables: 

Cost,  Quality,  Schedule,  Business  Relationship,  and  Subcontracting.  Each  variable 

receives  a  rating  using  Likert-style  responses:  1  =  Unsatisfactory,  2  =  Marginal,  3  = 
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Satisfactory,  4  =  Very  Good,  and  5  =  Excellent.  In  this  research,  we  used  an  overall 
average  of  CPARS  ratings  as  the  second  DV  (i.e.,  we  combined  each  variable’s  CPARS 
rating  to  make  an  average  CPARS  rating  for  each  contract). 

Our  model  contained  one  independent  variable  (IV).  A  researcher  may  use  an  IV 
to  induce  a  change  in  an  outcome,  or  DV.  In  our  case,  the  IV  equates  to  the  source 
selection  strategy  used  for  the  contract:  LPTA  or  tradeoff.  An  integrated  product  team, 
specifically  the  Contracting  Officer,  will  choose  the  preferred  source  selection  strategy. 
The  IV  is  assigned  LPTA  or  tradeoff  (LPTATO)  and  is  a  binary  variable  where  0  = 
LPTA  and  1  =  tradeoff. 

Lastly,  our  analysis  contained  four  covariate  variables.  Covariates  are  secondary 
variables  that  can  affect  the  relationship  of  primary  interest.  Essentially,  covariate 
variables  affect  the  relationship  between  the  IV  and  the  DV  and  are  variables  other  than 
the  IV  that  possibly  affect  the  outcome  variable,  or  DV.  Our  initial  list  of  covariates 
included  (a)  contract  dollar  value  (VALUE),  (b)  number  of  evaluation  factors 
(NUMEVALF ACTORS),  (c)  number  of  reviews  (NUMREVIEWS),  and  (d)  number  of 
offers  (NUMOFFERS).  All  four  covariates  are  continuous  variables.  We  intend  to 
remove  the  effects  of  the  covariates  in  hopes  of  seeing  the  effects  a  source  selection 
strategy  (LPTATO)  has  on  the  outcome  variables  (PALT  or  CPARS  ratings). 

C.  DESCRIPTIVE  STATISTICS 

Figure  9  shows  the  basic  descriptive  statistics  for  each  variable  and  indicates 
descriptive  statistics  for  (a)  all  the  data,  (b)  LPTA  source  selections,  and  (c)  tradeoff 
source  selections. 
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Descriptive  Statistics 

Bold=total  for  all  data,  non-italicized=LPTA,  italicized=tradeoff 


Variable 

Obs 

Mean 

SD 

Min 

Max 

PALT  (days) 

24 

447.61 

300.86 

123 

1269 

4 

277.50 

114.17 

167 

398 

13 

524.15 

318.70 

171 

1269 

Average 

CPARS 

Rating 

12 

3.54 

.73 

2.60 

5.00 

1 

3.00 

0.00 

3.00 

3.00 

6 

3.37 

.78 

2.60 

4.80 

Average 

Dollar 

Value 

24 

$73,500,000 

$106,000,000 

$747,736 

$386,000,000 

5 

$15,700,000 

$32,000,000 

$747,736 

$72,900,000 

13 

$82,100,000 

$122,000,000 

$982,898 

$386,000,000 

Number  of 

Evaluation 

Factors 

17 

3.35 

1.66 

1.00 

6.00 

4 

1.00 

0.00 

1.00 

1.00 

13 

4.08 

1.12 

2.00 

6.00 

Number  of 

Reviews 

16 

3.19 

2.88 

1.00 

13.00 

2 

1.00 

0.00 

1.00 

1.00 

12 

3.50 

3.21 

1.00 

13.00 

Number  of 

Offers 

24 

1.71 

.99 

1.00 

4.00 

5 

1.00 

0.00 

1.00 

1.00 

13 

2.31 

1.03 

1.00 

4.00 

Figure  9.  Descriptive  Statistics  for  All  Data,  LPTA,  and  Tradeoff 


D.  DATA  ISSUES 

Upon  review  of  the  descriptive  statistics,  it  is  clear  that  we  had  too  few  LPTA 
cases.  For  the  LPTA  source  selection  strategy,  our  data  consisted  of  four  PALT  and  one 
CPARS  rating.  For  the  tradeoff  source  selection  strategy,  our  data  consisted  of  13  PALT 
and  six  CPARS  ratings.  Power  calculations  (a  =  .05,  p  =  .80)  suggest  we  would  have 
needed  1 1  PALT  cases  and  28  CPARS  cases  for  each  source  selection  strategy  to  achieve 
an  acceptable  statistical  power.  We  were  clearly  lacking  in  both  PALT  and  CPARS  cases. 
Furthermore,  the  data  were  substantially  unbalanced  regarding  the  number  of  cases  for 
each  source  selection  strategy  (five  LPTA  cases  and  19  tradeoff  cases).  An  unbalanced 
design  will  lead  to  uncertainty  about  the  mean  as  the  intercept  and  will  make  designation 
of  sums  of  squares  more  complicated.  As  an  alternative  solution,  however,  a  weighted 
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mean  may  be  employed  as  a  substitute  to  the  grand  mean,1  and  the  Stata  software  will 
inevitably  control  the  assignment  of  the  sums  of  squares. 


E.  METHODOLOGY 

Our  intention  was  to  apply  a  technique  called  multivariate  analysis  of  covariance, 
or  MANCOVA,  to  evaluate  differences  in  contract  outcomes  (PALT  and  CPARS  ratings) 
based  on  contracting  methodology  (LPTA  or  tradeoff).  MANCOVA,  a  group  comparison 
method,  is  a  technique  that  generates  a  new  dependent  variable  using  information  from 
our  dependent  variables  (PALT  and  CPARS  ratings).  This  new  dependent  variable 
exploits  differences  between  the  grouping  variable  (LPTA  or  tradeoff  source  selections). 
However,  the  data  failed  to  meet  many  of  the  assumptions  required  for  MANCOVA 
because  of  the  lack  of  cases.  Similarly,  we  could  not  examine  the  DVs  separately  using 
the  univariate  method  of  analysis  of  covariance  (ANCOVA)  due  to  the  lack  of  cases. 

With  our  limited  data,  we  performed  a  multiple  regression  using  PALT  as  the  DV 
to  assess  whether  or  not  NUMREVIEWS  (the  new  IV)2  and  NUMOFFERS  (covariate) 
significantly  affected  the  PALT  for  tradeoff  source  selections.  Unfortunately,  we  had  to 
drop  VALUE  and  NUMEVALF ACTORS  because  they  did  not  pass  the  linearity 
assumption.  However,  we  must  caution  the  reader  that  there  was  not  enough  statistical 
power  to  interpret  the  results  with  confidence. 

F.  ASSUMPTION  TESTING 

Prior  to  conducting  the  multiple  regression,  we  assessed  certain  assumptions 
concerning  the  data.  First,  we  looked  for  any  missing  cases  and  located  one  case  that  was 
missing  the  NUMREVIEWS  variable.  We  chose  to  delete  the  case  using  listwise  deletion 
(n  =  12).  Second,  we  looked  for  outliers  using  plots  of  the  DV  against  the  IV  and 
covariate — one  outlier  on  the  NUMREVIEWS  variable  was  found  and  subsequently 
dropped  (n  =  11).  Third,  we  examined  the  normality  of  each  of  the  variables.  PALT, 

1  The  grand  mean  is  the  intercept  in  a  balanced  design. 

2  NUMREVIEWS  is  now  considered  the  IV,  as  it  can  be  manipulated  by  naval  contracting  policies. 
Previously,  we  made  the  assumption  that  the  policies  are  not  up  for  revision.  NUMOFFERS  remains  a 
covariate  variable  that  may  affect  PALT  independently  of  the  IV. 
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NUMREVIEWS,  and  NUM OFFERS  all  received  a  logarithmic  transformation,  which 
significantly  improved  their  positive  skewness.  Fourth,  we  assessed  linearity  between  the 
variables.  Linearity  was  not  a  problem  for  NUMREVIEWS  and  NUMOFFERS,  but  it  is 
important  to  keep  in  mind  that  we  have  a  very  small  number  of  cases,  so  there  are  few 
data  points  to  determine  linearity.  Finally,  we  checked  for  multicollinearity,  which  is  not 
an  issue. 

With  all  assumptions  tested,  we  perform  a  multiple  regression  between  PALT  as 
the  DV,  NUMREVIEWS  as  the  IV,  and  NUMOFFERS  as  the  covariate.  We  used  only 
tradeoff  cases,  as  an  insufficient  number  of  LPTA  cases  existed  for  this  analysis. 

G.  RESULTS 

Using  PALT  as  the  DV,  the  results  showed  that  the  model  is  not  significant 
(F(2,8)  =  2.56,  p  =  .14).  Neither  NUMREVIEWS  (bx  =  .42,  ns)  or  NUMOFFERS  (b2  = 
.65,  ns)  significantly  affected  time-to-contract  in  tradeoff  source  selections.  These  results 
were  not  surprising,  because  even  though  the  tradeoff  strategy  contained  our  largest  pool 
of  data,  there  was  an  insufficient  number  of  cases  to  achieve  statistical  significance.  More 
data  are  required  to  adequately  test  our  hypotheses. 

H.  SUMMARY 

The  chapter  identified  previous  research  teams  that  have  contributed  to  this 
overall  research  effort  and  provided  an  overview  of  the  research  methodology  used  to 
answer  our  research  questions.  We  identified  and  provided  a  basic  description  of  the 
variables  used  in  our  data  analysis,  addressed  assumptions  and  discussed  data  issues,  and, 
finally,  delivered  our  results.  Chapter  V  provides  a  summary,  our  conclusion,  and 
recommendations  for  further  research. 
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V.  SUMMARY,  CONCLUSION,  AND  RECOMMENDATIONS 

FOR  FURTHER  RESEARCH 

In  this  chapter  we  present  a  summary  of  our  research,  conclusions  to  our  research 
questions,  and  recommendations  to  guide  future  research  efforts. 

A.  SUMMARY 

Current  and  future  budget  realities  are  compelling  contracting  professionals  to 
become  more  efficient  in  their  procurement  efforts.  Developing  an  acquisition  strategy 
based  on  sound  procurement  planning  is  one  of  the  most  important  aspects  of  the  contract 
management  process.  The  source  selection  strategy — be  it  LPTA,  tradeoff,  or  a  variation 
of  the  two  along  the  best  value  continuum — should  ensure  that  “the  expected  outcome  of 
an  acquisition  ...  provides  the  greatest  overall  benefit  in  response  to  the  requirement” 
(FAR  2.101).  Recent  GAO  reports  signal  DOD’s  increased  frequency  of  employing 
LPTA,  and  some  in  the  defense  industry  are  concerned  that  an  increase  in  LPTA  will 
undermine  industry  innovation  and  companies  will  focus  on  cutting  costs.  To  alleviate 
industry  concerns,  Schultz  and  Dotson  encourage  contracting  professionals  to  use  a  more 
holistic  approach  that  considers  an  organization’s  landscape,  the  requirement’s  mission 
and  operational  interfaces,  industrial  capabilities,  contract  deliverables,  risk,  and 
opportunity  management  when  determining  an  appropriate  source  selection  strategy.  The 
purpose  of  this  research  was  to  explore  the  possible  relationships  between  the  contract 
source  selection  strategy  and  pre-award  and  post-award  performance  metrics.  Data  were 
collected  from  contract  files  at  NAVSEA.  NAVSEA  was  chosen  because  it  is  the  Navy’s 
largest  procurement  organization,  buying  complex  sea  systems  unique  to  the  Navy 
through  the  same  contract  management  processes  used  throughout  the  DOD  and  federal 
government.  Our  data  were  analyzed  to  determine  if  relationships  exist  between  source 
selection  strategies  and  performance  outcomes. 
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B.  CONCLUSIONS 


The  objective  of  our  research  was  to  determine  if  source  selection  strategy  affects 
pre-  and  post-award  metrics.  Specifically,  as  stated  in  Chapter  I,  our  research  intended  to 
answer  the  following  questions: 

1.  How  does  the  source  selection  strategy  affect  pre-award  metrics  (e.g., 
PALT,  number  of  solicitation  amendments  or  protests)? 

Despite  the  data  collected  to  answer  this  question,  they  were  insufficient  for 
statistical  analysis.  The  sample  size  was  not  numerically  sufficient  to  complete  the 
desired  analysis.  However,  we  observed  a  positive  correlation  between  average  dollar 
value  and  source  selection  strategy.  Within  our  small  sample,  a  higher  dollar  value 
seemed  to  be  associated  with  the  use  of  the  tradeoff  source  selection  strategy.  Similarly, 
a  positive  correlation  between  PALT  and  source  selection  strategy  seems  to  suggest  that 
longer  PALTs  are  associated  with  the  tradeoff  strategy.  This  is  intuitive,  as  the  tradeoff 
strategy  is  typically  used  in  procurements  with  requirements  that  are  difficult  to  define  or 
allow  for  multiple  solutions.  In  addition,  given  the  multitude  of  non-cost  factors 
evaluated  using  the  tradeoff  strategy,  the  development  of  offers  and  the  source  selection 
process  itself  are  likely  to  demand  more  time  than  the  more  objective,  price-focused 
LPT  A  strategy.  The  higher  complexity  of  the  procurements  using  the  tradeoff  strategy 
seems  to  explain  the  association  with  a  longer  PALT. 

2.  How  does  the  source  selection  strategy  affect  post-award  ratings  (CPARS 
and  EVM  performance  metrics)? 

Despite  the  data  collected  to  answer  this  question,  they  were  insufficient  for 
statistical  analysis.  Thus,  we  were  not  able  to  answer  the  questions  we  sought  to  answer 
at  the  beginning  of  this  research.  Our  research  has,  however,  increased  the  data  pool  and 
refined  the  methodology  for  future  research  teams.  We  provide  recommendations  for 
future  research  in  the  next  section. 

C.  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

Our  research  project  is  part  of  an  ongoing  research  initiative  that  requires 

additional  data.  Previous  research  teams  have  recommended  that  future  researchers 

gather  more  EYM  and  CPARS  data  to  further  this  research  initiative.  These  generalized 
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recommendations  led  us  to  believe  that  we  needed  to  locate  high  dollar  value  contracts 
that  would  trigger  CPARS  and  EVM  reporting  requirements  and  provide  us  with 
historical  documentation  in  the  contract  file.  This  assumption  was  incorrect.  We 
identified  contracts  that  were  required  to  document  contractor  performance  and  utilize 
EVM  reporting,  but  as  we  reviewed  the  contract  files,  we  discovered  that  CPARS  and 
EVM  reports  were  not  located  in  the  archived  contract  file.  However,  after  we  completed 
our  onsite  research  visit  to  NAVSEA,  we  discovered  that  PARC  A  established  an  EVM 
centralized  repository  (EVM-CR)  in  2007  that  maintains  EVM  data  on  Acquisition 
Category  (ACAT)  1A,  1C,  ID,  and  other  high  visibility  acquisition  programs.  Based  on 
this  new  information,  we  make  the  following  recommendations. 

1.  Recommendation  #1 

Prior  to  selecting  a  contracting  organization  to  conduct  onsite  research,  we 
recommend  that  future  research  teams  obtain  access  to  the  PARCA  EVM-CR  and 
identify  a  unit  identification  code  (UIC)  that  contains  an  adequate  sample  size  of 
contracts  that  contain  EVM  data  to  support  additional  research.  By  doing  so,  future 
research  teams  will  be  able  to  gather  one  of  the  most  challenging  data  points  first  and 
work  through  predetermined  contract  files  to  locate  the  remaining  necessary  data. 

2.  Recommendation  #2 

Once  contracts  are  identified  through  the  PARCA  EVM-CR,  we  recommend 
contacting  the  procurement  organization  that  manages  the  contracts  and  requesting  access 
and  approval  to  conduct  onsite  research.  If  approved,  we  recommend  forwarding  the 
contract  numbers  to  the  organization's  CPARS  program  analyst.  The  CPARS  program 
analyst  will  be  able  to  generate  CPARS  data  reports  useful  to  the  research  project  and 
may  identify  information  missing  from  the  report.  During  our  research,  we  found  that 
some  CPARS  reports  were  not  properly  documented  and  that  program  managers  (PMs) 
and  contracting  officer  representatives  (CORs)  contact  information  was  missing.  Being 
able  to  identify,  prior  to  an  onsite  visit,  which  contracts  are  missing  PM  and/or  COR 
contact  information  will  enable  future  research  teams  to  identify  PM  and  COR  contact 
information  from  the  archived  contract  files.  Once  PM  and  COR  contact  information  is 
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obtained,  we  recommend  future  research  teams  contact  each  PM  and/or  COR  to  obtain 
the  necessary  CPARS  data  points. 

To  assist  our  advisors  in  further  research,  we  have  provided  50  contract  numbers 
from  NAYSEA  that  contain  EVM  data  and  recommend  that  those  numbers  be  turned 
over  to  future  research  teams.  We  also  recommend  that  future  research  teams  do  not 
underestimate  the  size  and  number  of  volumes  many  of  the  contract  files  contain  and  the 
time-intensive  nature  of  locating  specific  data  points  for  the  Excel  data  guide.  We  also 
recommend  that  research  teams  do  not  visit  organizations  that  do  not  employ  a  sufficient 
mixture  of  LPTA  and  Tradeoff  source  selection  strategies.  For  example,  Fleet  Logistic 
Center  Norfolk  typically  procures  simple,  low  dollar  value  commodities  that  do  not 
provide  ample  data  points  and  do  not  contribute  to  the  overall  research  initiative.  We 
hope  our  recommendations  prove  useful  to  future  research  teams  and  that  they  will  be 
able  to  use  our  methodologies  to  obtain  adequate  statistical  power  to  test  the  relationship 
between  source  selection  strategies  and  performance  outcomes. 
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APPENDIX.  DATA  COLLECTION  CATEGORIES 


Basic  Info 

Observation  # 

Service 

Organization 

Contract  Number 

Name 

NAICS 

PSC 

Acquisition  Complexity 

Predominant 

Contract 

Type 

Sole  Source 

SB  Set  Aside 

Consoidated  Requirement 

Multiyear 

#  Years 

Dollar  Value 

Trade  Off  or 

LPTA 

#  Evaluation 

Factors 

Eval  Factor 

#1 

#  Subfactors  inEF#l 

Eval  Factor  #2 

#Subfactors 

in  EF#2 

Eval  Factor 

#3 

#  Subfactors 

in  F.F#3 

Eval  Factor 

#4 

#  Subfactors 

in  EF#4 

Eval  Factor  #5 

#  Subfactors  inEF#5 

# 

Subelements 

ACAT/SCAT 

Level 

FAR  Part 

Life  Cycle 
Stage 

Joint  Urgent 
Operational 
Need 
(JUON) 

#  CLINs 

Pre -Solicit  at  ion  Conference/  Industry  Day 

Pre-Proposal 

Conference 

Pre- 

AwardPrc- 

Performance 

Conference 

#  Offerors 

#  Competitive 
Range 

Single  or 
Mult^le 
Award 

On-Ramp  Provision 

Incentive  Fee  Criterion  #1 

Incentive  Fee 

Criterion  #2 

Incentive  Fee 

Criterion  #3 

Incentive  Fee 

Criterion  *4 

Incentive  Fee 

Criterion#? 

Award  Fee 

Criterion#! 

Award  Fee  Criterion  #2 

Award  Fee  Criterion  #3 

Award  Fee 

Criterion  #4 

Award  Fee 

Criterion  #5 

Environmental  Factors 

#  Reviews 

Review  1 

Date 

Review  2 

Date 

Review  3  Date 

Review  4  Date 

#  Personnel 

on  Source 

Selection 

Team 

Source 

Selection 

Authority 

#  Team 

Locations 

ACE  Support 

MIRT 

Support 

Use  of  Collaboration  Tools 

Abundance/  Lack  of  File  Documentation  Rating 

State  of  the 

Economy 

Outcome  Variables 

PALT  - 
Requirement 
Receipt  Date 

PALT  -  RFP 

Issue  Date 

PALT- 

Proposab 

Received 

Date 

PALT  -  Contract  Award  Date 

PALT  -  Days 

Early 

Strategy  and 
Issues 

Session 
(ESIS)  Date 
(>$100M) 

ASP  Date 

Protest 

Protest  Level 

Protest 

Results 

Corrective  Action  (Describe) 

CPAR  Cost  Control 

CPAR 

Technical 
(quality  of 
product  or 
service) 

CPAR 

Schedule/ 

Timeliness 

CPAR  Mgt 
of  Business 
Relationship 

SB  Subktng 
(if  applicable) 

EVM  CV 

EVM  SV 

EVM  CPI 

EVM  SPI 

Nunn- 

McCurdy 

Breach 

#  of  RFP 

Amendments 

RFP 

Amendment 

Reason  1 

RFP 

Amendment 

Reason  2 

RFP  Amendment  Reason  3 

RI  P  Amendment  Reason  4 

RFP 

Amendment 

Reason  5 

RFP 

Amendment 

Reason  6 

Earned 

Award  Fee 

% 

Earned  Fee 

Amount/ 

Target  Fee 

Other  Variables 

#  Evaluation 

Notices 

#  ENs  Prc- 
Campetkive 
Range 

#  ENs  Post- 
Competitive 
Range 

#  Rounds  of  ENs 

#  Clarification  Requests 

# 

Deficiency 
Re  pons 

Basis  for 

Award 
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